Skip to content

fix(url_safety): evaluate embedded IPv4 for NAT64 64:ff9b::/96 addresses - #38058

Open
liuhao1024 wants to merge 1 commit into
NousResearch:mainfrom
liuhao1024:fix/url-safety-nat64-dns64-prefix
Open

fix(url_safety): evaluate embedded IPv4 for NAT64 64:ff9b::/96 addresses#38058
liuhao1024 wants to merge 1 commit into
NousResearch:mainfrom
liuhao1024:fix/url-safety-nat64-dns64-prefix

Conversation

@liuhao1024

@liuhao1024 liuhao1024 commented Jun 3, 2026

Copy link
Copy Markdown

What does this PR do?

Fixes a false-positive SSRF block in tools.url_safety._is_blocked_ip() that prevents normal public websites from being fetched when the DNS resolver returns DNS64/NAT64-synthesized AAAA records in the 64:ff9b::/96 well-known prefix.

Related Issue

Fixes #38048

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • tools/url_safety.py: Add _NAT64_WKP network constant and NAT64 handling in _is_blocked_ip() — decode the embedded IPv4 from the low 32 bits and run standard IPv4 safety checks instead of blocking the entire prefix as is_reserved.
  • tests/tools/test_url_safety.py: Add parametrized tests for NAT64 blocked IPs (private/metadata embedded targets) and allowed IPs (public embedded targets), plus integration tests for is_safe_url with NAT64 addresses.

How to Test

  1. Run pytest tests/tools/test_url_safety.py -v — all 126 tests should pass including the 12 new NAT64 tests
  2. Verify that 64:ff9b::6812:27e4 (embeds 104.18.39.228, a public Cloudflare IP) is allowed by _is_blocked_ip()
  3. Verify that 64:ff9b::a9fe:a9fe (embeds 169.254.169.254, AWS metadata) is blocked by _is_blocked_ip()
  4. Verify that 64:ff9b::0a00:0001 (embeds 10.0.0.1, private) is blocked by _is_blocked_ip()

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/tools/test_url_safety.py -q and all 126 tests pass
  • I've added tests for my changes (12 new NAT64 test cases)
  • I've tested on my platform: macOS

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Code Intelligence

  • Analyzed: tools/url_safety.py:_is_blocked_ip (called by is_safe_url, is_always_blocked_url — SSRF protection entry points)
  • Blast radius: LOW — additive guard for a specific IPv6 prefix; existing IPv4 and IPv4-mapped paths unchanged
  • Related patterns: mirrors the existing IPv4-mapped IPv6 handling pattern (lines 151-158); 198.18.0.0/15 benchmark range handled by existing is_reserved check in PR fix(url_safety): exclude 198.18.0.0/15 benchmark range from SSRF private-IP blocking #35436

@liuhao1024
fix(url_safety): evaluate embedded IPv4 for NAT64 64:ff9b::/96 addresses
600a629 
DNS64/NAT64-synthesized AAAA records in the 64:ff9b::/96 well-known
prefix (RFC 6052) embed a public IPv4 address in the low 32 bits.
Python marks the entire prefix as `is_reserved`, which caused
`_is_blocked_ip()` to false-positive block normal public sites when
the resolver returns synthesized AAAA records alongside A records.

Add explicit handling analogous to the existing IPv4-mapped IPv6 path:
decode the embedded IPv4 and run the standard IPv4 safety checks on it.
This preserves SSRF protection (e.g. 64:ff9b::a9fe:a9fe →
169.254.169.254 is still blocked) while allowing public targets
(e.g. 64:ff9b::6812:27e4 → 104.18.39.228 passes).

Fixes NousResearch#38048
@alt-glitch alt-glitch added type/bug Something isn't working tool/web Web search and extraction type/security Security vulnerability or hardening P2 Medium — degraded but workaround exists labels Jun 3, 2026
@alt-glitch alt-glitch mentioned this pull request Jun 3, 2026
Closed
@liuhao1024 liuhao1024 mentioned this pull request Jun 8, 2026
Open
teknium1
teknium1 reviewed Jul 13, 2026
View reviewed changes

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for preserving the embedded-IPv4 checks rather than blanket-allowing the NAT64 prefix; that matches the related issue’s hardening requirement.

Problems

  • The new branch only runs in _is_blocked_ip(). On current main, is_safe_url() skips that helper when security.allow_private_urls is enabled (tools/url_safety.py:443). Its preceding always-blocked check only compares the raw NAT64 wrapper to _ALWAYS_BLOCKED_IPS / _ALWAYS_BLOCKED_NETWORKS (tools/url_safety.py:435-441), so a NAT64 wrapper around IMDS is not covered under that toggle.
  • is_always_blocked_url() has the same raw-address-only sentinel checks at tools/url_safety.py:329-370 and is not changed here.

Suggested changes

  • Reuse a NAT64 embedded-IPv4 helper for both ordinary and always-blocked policy checks.
  • Add toggle-enabled and is_always_blocked_url() regressions for NAT64-wrapped metadata/link-local targets; the existing suite establishes that metadata must remain blocked despite the toggle (tests/tools/test_url_safety.py:445-503).

Automated hermes-sweeper review.

Comment thread tools/url_safety.py
# itself is ``is_reserved`` in Python, but the embedded IPv4 target
# may be a perfectly public host. Evaluate the embedded IPv4
# instead of blocking the NAT64 wrapper wholesale.
if isinstance(ip, ipaddress.IPv6Address) and ip in _NAT64_WKP:

@teknium1 teknium1 Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This protects the ordinary _is_blocked_ip() path, but is_safe_url() skips that helper when security.allow_private_urls is enabled. Please also decode NAT64 before the always-blocked metadata checks so an embedded IMDS address remains blocked under the toggle.

@teknium1 teknium1 added sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 14, 2026
@zavrenn

zavrenn commented Jul 25, 2026

Copy link
Copy Markdown

The floor half is still open here. is_safe_url(), is_always_blocked_url() and _resolved_http_connect_ips() all compare the raw resolved address against _ALWAYS_BLOCKED_IPS / _ALWAYS_BLOCKED_NETWORKS, which only hold plain-IPv4 and ::ffff: forms — so a NAT64-wrapped metadata address matches nothing there. Right now is_reserved catches it incidentally; once this lands, allow_private_urls skips that catch, and that toggle is the workaround DNS64 operators are using meanwhile.

Suggest extracting the unwrap into a helper covering both ::ffff: and 64:ff9b:: and routing the floor checks through it too. I have that implemented against main with 30 regression tests if it's useful.

zavrenn added a commit to zavrenn/hermes-agent that referenced this pull request Jul 25, 2026
@zavrenn @claude
fix(url_safety): unwrap NAT64/DNS64 addresses in the SSRF guard
38aefd2 
IPv6-only containers resolve through DNS64, which answers A-only hostnames
with a synthesized AAAA in the RFC 6052 well-known prefix, carrying the real
IPv4 in the low 32 bits (64:ff9b::d4d9:2e0a = public 212.217.46.10).
ipv4_mapped is None for these, and Python's ipaddress reports all of
64:ff9b::/96 as is_reserved, so _is_blocked_ip fell through to the generic
IPv6 branch and blocked every DNS64-resolved host — an agent in such a
container had no outbound web access at all.

Add a shared _embedded_ipv4() helper covering both IPv6 wrappers a resolver
can return (::ffff:x.x.x.x and 64:ff9b::x.x.x.x) and judge the embedded IPv4
by the full IPv4 ruleset, so 64:ff9b::7f00:1 (loopback) and 64:ff9b::c0a8:1
(private) stay blocked. Only the /96 well-known prefix is unwrapped; custom
NAT64 prefixes place the IPv4 at a prefix-dependent offset and are left
blocked rather than guessed at.

Also route the always-blocked metadata floor through the same unwrap. The
floor compared raw resolved IPs against sets holding only plain-IPv4 and
::ffff: forms, so 64:ff9b::a9fe:a9fe (169.254.169.254) matched nothing and
was blocked only incidentally by is_reserved — which allow_private_urls
skips. Unwrapping in _is_blocked_ip alone would turn that incidental block
into a real gap.

Upstream: reported as NousResearch#38048, partially fixed by PR NousResearch#38058 (in-flight, covers
_is_blocked_ip only). Drop this carry once a version landing both halves is
merged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@GottZ GottZ mentioned this pull request Aug 3, 2026
Open
1 task
@egilewski

egilewski commented Aug 11, 2026

Copy link
Copy Markdown

suggesting changes

Security evidence:

  • trust boundary: DNS resolver answers feed both the URL preflight check and direct HTTP connection; a NAT64 answer carries its routable IPv4 destination in the low 32 bits.
  • source/sink/invariant: The new decoding correctly permits public NAT64 addresses and rejects private embedded addresses by default, but the cloud-metadata/link-local floor must remain unconditional when private-URL access is opted out.
  • current-main reproduction: Current main rejects a public NAT64 answer as reserved, while default metadata blocking still relies on the ordinary private-address check.
  • PR-head or patch-replay validation: The PR/replay permits public NAT64, but with private-URL access enabled it accepts a NAT64 wrapper for 169.254.169.254 through URL validation and connect-time resolution, and the dedicated always-blocked check does not identify it.
  • positive/negative cases: Public and private NAT64 mappings behave correctly by default, and IPv4-mapped metadata remains blocked; the opt-out-enabled NAT64 metadata case bypasses the floor.
  • residual bypass search: The same raw-address floor is used by the browser-facing floor check and the direct HTTP connect validator, so this is reachable through multiple URL sinks.
  • reviewer validation: Focused URL-safety and adjacent browser/media SSRF tests pass; NAT64 opt-out, floor, and connect-time regressions are not covered.

Please normalize NAT64 (alongside the existing IPv4-mapped form) before every always-blocked decision and add opt-out-enabled preflight, floor, and connect-time regressions while retaining the public-NAT64 positive case.

Not checked:

  • Full test suite

Signed: GPT-5.6-sol-xhigh in Codex

This was referenced Sep 20, 2026
Closed
Closed

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@teknium1 teknium1 teknium1 left review comments

Assignees

No one assigned

Labels

P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data tool/web Web search and extraction type/bug Something isn't working type/security Security vulnerability or hardening

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

url_safety false-positively blocks DNS64/NAT64 addresses in 64:ff9b::/96

5 participants

@liuhao1024 @zavrenn @egilewski @teknium1 @alt-glitch